For this, we utilize an auxiliary contrastive model with the popular heuristic that biased features are learned preferentially during training.

Image classification models tend to make decisions based on peripheral attributes of data items that have strong correlation with a target variable (i.e., dataset bias). These biased models suffer from the poor generalization capability when evaluated on unbiased datasets. Existing approaches for debiasing often identify and emphasize those samples with no such correlation (i.e., bias-conflicting) without defining the bias type in advance. However, such bias-conflicting samples are significantly scarce in biased datasets, limiting the debiasing capability of these approaches. This paper first presents an empirical analysis revealing that training with diverse bias-conflicting samples beyond a given training set is crucial for debiasing as well as the generalization capability. Based on this observation, we propose a novel feature-level data augmentation technique in order to synthesize diverse bias-conflicting samples. To this end, our method learns the disentangled representation of (1) the intrinsic attributes (i.e., those inherently defining a certain class) and (2) bias attributes (i.e., peripheral attributes causing the bias), from a large number of bias-aligned samples, the bias attributes of which have strong correlation with the target variable. Using the disentangled representation, we synthesize bias-conflicting samples that contain the diverse intrinsic attributes of bias-aligned samples by swapping their latent features. By utilizing these diversified bias-conflicting features during the training, our approach achieves superior classification accuracy and debiasing results against the existing baselines on both synthetic and real-world datasets.

It is particularly challenging to address this problem when access to bias labels is not permitted.

However, spurious correlation remains an ongoing challenge, primarily due to the difficulty in precisely detecting these samples.

This supplementary material presents additional results and descriptions of our approach that are not included in the main paper due to the page limit. Afterwards, we illustrate the implementation details including architecture designs and hyper-parameters for training in Section D. Lastly, Section E We again observe the superiority of our method regardless of the corruption types.Dataset Ratio (%) LfF [1] Ours Corrupted CIFAR-10 Type 0 0.5 33.95 Table 1: Image classification accuracy evaluated on unbiased test sets of Corrupted CIFAR-10 Type 0 and Type 1 with varying ratio of bias-conflicting samples. Best performing results are marked in bold. Similar to Figure 1 of the main paper, columns and rows correspond to those images where the bias attribute ( i.e., gender) and the intrinsic attribute ( i.e., age) are extracted, As mentioned in Section 5 of the main paper, we define'age' as either'young' or'old' In addition, we observe that the ages of reconstructed images change as the row changes. We also provide training details of the decoder for BFFHQ in Section D.4.

Image classification models tend to make decisions based on peripheral attributes of data items that have strong correlation with a target variable ( i.e., dataset bias). These biased models suffer from the poor generalization capability when evaluated on unbiased datasets.

To be more specific, workers were asked to answer three binary questions.

Neural networks often learn to make predictions that overly rely on spurious correlation existing in the dataset, which causes the model to be biased .

We mark the adopted value in bold . We study the impact of training randomly sampled subset, learning by committee, and transferring the knowledge of the main classifier. We mark the best performance in bold . We mark the best performance in bold . We extend Table 5 of the main paper.